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Abstract 


As school choice options grow, it is helpful to test for indirect effects on non-choosers who are left behind 
in district-run schools. Can a competitive system built on the principle of choice serve as a rising tide that 
lifts all boats or will such systems further existing inequities? The largest competitive effects analysis of a 
voucher program in the United States, this study helps address existing gaps in knowledge by examining 
how the Indiana Choice Scholarship Program affects students in traditional public schools, looking at the 
change in academic outcomes and graduation rates of students in Indiana traditional public schools based 
on their proximity to private schools participating in this statewide K—12 private school voucher program. 
We use network proximity data to generate a “drive-time” measure that accounts for road lengths, 
intersection turn times, speed limits, and traffic data and allows us to calculate precise travel time between 
two points—down to the minute. We use this precise measure in a difference-in-differences framework to 
examine impacts on student math and English Language Arts test scores and graduation rates. We find 
little evidence that the average student in traditional public schools has been affected—ceither positively or 
negatively— by the enactment and growth of the Indiana Choice Scholarship Program. We do find, 


however, consistent evidence of small positive effects for low-income children. 


Keywords: School choice; education policy; market reforms 


Introduction 


Passed as part of House Enrolled Act 1003-2011, the Indiana Choice Scholarship Program (ICSP) 
is a means-tested private school choice program that currently provides $159 million in state funds 
annually for more than 36,000 students to offset the cost of private school tuition. The largest voucher 
program in the nation, enrollment in the program has grown every year since the program was first 
implemented in the 2011-12 school year (Table 1). During the program’s first year of operation, ICSP 
scholarships were limited to 7,500 students. Student participation fell short of the cap that year when 
3,911 students participated, receiving an average award of $4,105 (Grades 1—8) or $5,518 (Grades 9-12) 
each. The program participation cap doubled in the program’s second year and for the 2013-14 school 
year, the enrollment cap was removed altogether so that ICSP scholarships could be awarded to all 
eligible applicants. By 2019-20, enrollment had reached 36,707, with eligible students receiving an 
average award of $5,900 (grades 1-8) or $5,909 (Grades 9-12).! 

One of the main reasons for this rate of program growth is the expansion of program eligibility 
criteria over time to include a broader cross-section of students. The program is currently open to 
applicants with special educational needs, siblings of current ICSP scholarship users, and students coming 
from failing public schools, in addition to children from low-income families. Although there was one in 
the first few years of the program, there is currently no enrollment cap on participants. Along with 
existing evidence on participants’ test scores (Waddington & Berends, 2018), this raises important 
questions about the impact of a rapidly-growing private school choice program in the Hoosier State. 
Given the many pathways to entry and lack of enrollment cap, some may worry that students who don’t 
choose a private school will be left behind in more than one way. Can a competitive system built on the 
principle of choice serve as a rising tide that lifts all boats, or will such systems further existing 


inequities? 


' These are the award amounts for students receiving a full scholarship (what is deemed a “90% award” by the 
Indiana Department of Education). Partial scholarships were also available in these years (what is deemed a “50% 
award”). The average award for a student on a 50% award was $2,928 (Grades 1-8) or $2,954 (Grades 9-12) in 
2011-12. The average award for a student on a 50% award was $3,173.26 (Grades 1-8) or $3,202 (Grades 9-12) in 
2019-20. 


Table 1. Growth in the Indiana Choice Scholarship Program, 2011-12 through 2019-20 


Year Student Participation Private School Participation 
2011-12 3,911 241 
2012-13 9,139 289 
2013-14 19,809 313 
2014-15 29,148 314 
2015-16 32,686 316 
2016-17 34,299 313 
2017-18 35,458 318 
2018-19 36,290 329 
2019-20 36,707 326 


Source: Indiana Department of Education, Office of School Finance 


Various arguments have been proposed in support of the idea that the competitive pressures 
induced by an education marketplace will serve as a tide that lifts all boats. In response to the fiscal 
pressure to attract and retain students, schools may seek out innovative approaches to improve student 
learning. On the other hand, opponents argue a potential drawback is the possibility that non-choosers 
will be left behind, furthering inequitable schooling experiences. For example, the executive director of 
the Indiana Association of Public School Superintendents, J. T. Coopman, has said about the program: 
"There's a price that's going to be paid down the road because we're going to end up with the haves and 
the have-nots."(Wang, 2014). To test these theories, we conduct a systemic effects analysis of Indiana’s 
private school voucher program, one of the largest voucher programs in the nation. We analyze the 
ICSP’s competitive effect on the achievement and attainment outcomes of 837,000 students. 

To accomplish this, we rely on geocoded competition measures in a difference-in-differences 
framework, focusing on district-operated public schools as the population of interest. The intuition behind 
the research design is that while the establishment of the ICSP may have represented a shock for all 
public schools in Indiana, those facing a more competitive landscape to begin with would have felt this 
shock more acutely. 

First, we look at the competitive impact of the ICSP on student test scores in the program’s first 
year of operation (2011-12), when enrollment numbers were still small. This helps us cleanly isolate the 
competitive impact of the private school choice option from school compositional changes resulting from 
student transfers and resource changes associated with the loss of per-pupil funds, which occurs when 
there is an enrollment decline in the public-school sector. We then repeat the analysis incorporating five 
years of outcome data to maximize statistical power and to take advantage of higher “dosage” in later 
years when greater numbers of students exercised this choice option. These data include school years 


ending in 2011 (the pre-policy year), 2012 (the first post-policy year), 2013, 2014, 2015, and 2016. We 


also use this model to test for changes in public school students’ likelihood of graduation that could be 
attributed to the program. 

We use a variety of drive-time measures—the travel time in minutes to the nearest private school 
competitor and the average count of private school competitors within X minutes—to assess distance 
between schools. We calculate drive-time before the policy was enacted to avoid potential endogeneity 
bias that could occur if new private schools were established in response to the voucher program and 
systematically chose to establish near low-performing public schools, for example, where they might find 
it easier to recruit applicants. Because we take this caution, any observed changes in student achievement 
associated with the distance measure are likely to reflect a public school response to the competitive 
pressure associated with the passage of the voucher law. 

It may be the case that public schools only respond to the program once the competitive pressure 
they face passes beyond a certain threshold or they lose a meaningful number of students. We therefore 
conduct a dosage analysis of test score changes to search for a potential “tipping point” in the school 
choice marketplace. That is, we are interested in estimating if there is a point at which public school test 
scores change as a significant mass of students transfer from their assigned traditional public school to a 
private school of choice by way of the state’s private school choice program, the ICSP. To estimate this 
number, we calculate the competitive response associated with the drive-time to the nearest private school 
with a minimum of 10, 20 30, 40, or 50 voucher students in the program’s first year. 

In summary, the methods used for our competitive analysis of the nation’s largest voucher 
program employ multiple measures of competition, including drive-time to the nearest private school and 
a count of private school competitors within a given drive-time radius, each implemented in a separate 
model to avoid collinearity. The findings are as follows: null to small, positive impacts on public school 
achievement and null effects on a student’s likelihood of graduation. Specifically, drive-time models find 
no impact on student test scores, whereas competitor count models find small, positive, and statistically 
significant test score increases in both math and English Language Arts. 

We also test for subgroup impacts, finding very small but statistically significant math benefits 
for low-income and special education students in models that rely on the drive-time measure. 

The methods used for our tipping point analysis leverage the drive-time to private schools with a 
minimum count of students attending private schools by way of the ICSP to estimate impacts. The 
findings are as follows: a very small, positive, statistically significant achievement effect associated with 
the distance to a private school accepting at least 10 voucher students, but no additional response 
associated with distance to schools serving a greater mass of voucher students. 

The remainder of this study proceeds as follows. First, we review the theoretical framework 


underlying the competitive effects literature. We then place the ICSP in the context of the broader private 


school choice landscape. Next, we review prior studies and identify our contribution. The methods section 
describes our data and methods and is followed by a presentation of results. The article concludes with a 


discussion of the main findings and an acknowledgement of the limitations of the current study. 


Theoretical Framework 


Theoretically, the competitive impact of the ICSP on public school students’ test scores and 
likelihood of graduation could be positive, negative, or neutral. On the one hand, we may observe 
efficiency gains if public school leaders respond in creative and productive ways to the increased market 
pressure to achieve better academic outcomes at the same cost (Friedman, 1962). The theory of action by 
which schools would accomplish this might consist of some combination of the following reforms: a 
renewed focus on effective instructional leadership practices (Grissom, Loeb, & Master, 2013); better use 
of technology and strategic staffing (Jacob, 2011); overall efforts to boost organizational capacity and 
efficiency (Chubb & Moe, 1990), the development of human capital by way of teacher coaching (Kraft, 
Blazar, & Hogan, 2018), and revisions to school start schedules to better match teenagers’ sleep habits 
(Heissel & Norris, 2018). Even non-action on the part of public schools could result in the perception of 
achievement gains if the hardest-to-serve and lowest-performing students depart with a voucher, a 
scenario our research design will have to account for. 

On the other hand, we may observe lower achievement outcomes in traditional public schools in 
response to competition from private schools if, for example, public schools pursue incoherent curricular 
or pedagogical reforms or misuse resources. It is worthwhile spending some time to understand how 
school funding formulas contribute to potential outcomes in a competitive environment. In the absence of 
“hold-harmless” provisions in public school funding formulas, such as those enacted in response to the 
expansion of the charter school sector in Massachusetts (Ridley & Terrier, 2018), student transfers 
through Indiana’s voucher program automatically lower the total amount of funding received at the school 
level, even if the per-pupil funding amount remains unchanged. Because it has already been established 
there is a causal relationship between school spending and student achievement (Jackson, 2018), we 
might expect to observe a decline in traditional public school student achievement if public school 
budgets are dramatically impacted by the voucher program. If, however, school budgets are only 
marginally impacted by a small number of student transfers, or public schools are able to absorb 
reductions by cutting fixed costs to adapt to their new funding reality, students may be unaffected. 
Another avenue by which we could observe negative impacts on public schools is if private schools 


“cream skim” the brightest and most motivated students and families (Altonji, Huang, & Taber, 2015). 


This scenario would result in the perception of lower overall achievement scores even in the face of non- 
response to the competitive threat by public schools. 

Alternatively, we may observe neutral impacts on student outcomes. This might result from 
investment in cosmetic reforms that are not strongly associated with improvements to the instructional 
climate, such as creating brochures or applying fresh paint to the walls of the school facility. 

As a final theoretical note, it is worth observing that any initial changes to achievement or 
attainment outcomes may level off. This latter point about non-linearity is important and the motivation 
for us to examine the “tipping point” theory in addition to overall impacts. That is, it is unclear a priori if 
any achievement or attainment changes—ceither positive or negative—will be linearly related to 
competitive pressures because schools will likely respond to the program dynamically, adjusting their 
response to the voucher program in real time with changes to school practices, staffing, curriculum, in- 
service opportunities, and so on. 

Ultimately, the direction of effects is unclear. Therefore, our null hypothesis is that traditional 
public schools will demonstrate no change in academic outcomes in response to student transfers through 
Indiana’s private school voucher program. The alternative hypothesis is that traditional public-school 
achievement will either increase or decrease in response to competition from this targeted private school 


voucher program. 


The Indiana Choice Scholarship Program 
(ICSP) 


With 36,707 students participating in 2019-20, the ICSP is the largest statewide private school 
voucher program in the United States. Launched in 2011, there has been an eight-fold increase in student 
participation since its inception. The regulatory environment is unusual, but school participation remains 
high with 326 participating private schools in 2019-20.? In addition to common regulations such as the 
school accreditation requirement, non-discrimination rules, employee background checks, and compliance 
with health and safety codes, voucher-accepting private schools in Indiana are required to administer the 
battery of state tests of the state standards that is administered annually in public schools and report those 


data to the state, which then reports an A-F grade for all public schools and voucher-participating private 


? School participation counts for prior years are as follows: 2011-12: 241 schools; 2012-13: 289 schools; 2013-14: 
313 schools; 2014-15: 314 schools; 2015-16: 316 schools; 2016-17: 313 schools; 2017-18: 318 schools and 2018— 
19: 329 schools. 


schools based on combined test scores and graduation rate data. Participating private schools are subject 
to continuing eligibility regulations, including rules that they must allow the state to review their 
curriculum, observe classroom instruction, and review instructional materials and the stipulation that they 
cannot receive a D or F grade in two consecutive years. 

There are a number of features of program design that may generate meaningful competitive 
effects in Indiana. First, private schools receive accountability grades from the state, and those grades are 
made available to the general public. Unlike choice programs in other states, this practice makes it easy 
for parents, legislators, advocates, and members of the general public to draw comparisons across the 
public and private sectors and to make assessments about which particular schools are performing well. 
However, compelling private schools to teach and assess the same standards may limit differentiation and 
attenuate any potential effects of competition from the program. Second, the program serves a broad and 
diverse population, which means a greater segment of the general population is incentivized to pay 
attention to school performance and to consider switching sectors in search of better outcomes for their 
children. For children aged between five and 22, there are eight eligibility pathways, any one of which 
can qualify a student for a voucher. These various pathways cover students from low- and middle-income 
households (e.g., to receive a full scholarship in 2019-20, the annual household income cannot exceed 
$32,703 for a household of four people), students with special educational needs, students zoned to failing 
traditional public schools (defined as those receiving an F grade from the state), siblings of voucher 
recipients, and students who previously received an Early Education Grant to attend prekindergarten at a 


participating private school. 


Prior Literature 


In this section, we review the prior literature on this program, specifically, and the prior literature 


on the indirect impacts of private school choice nationally. 


Prior Literature on the ICSP 


Prior research on the direct impacts of the ICSP sheds light on program participants by examining 
changes in their test scores and parental satisfaction rates. Research on the achievement impacts for 
students who use vouchers in Indiana shows a modest negative impact in math and null effects in English 
Language Arts. Waddington and Berends (2018) exact-match on the following variables: race, sex, and 


baseline year, grade, and school. This matching strategy still results in meaningful differences between 


10 


the two groups at baseline, such as differences in prior test scores, however. To account for existing 
differences between the two groups under comparison, which may bias their results, they include prior 
achievement as a covariate in the ordinary least squares regression model used to estimate impacts. They 
report an average achievement loss of 0.15 standard deviations in mathematics during students’ first year 
of attending a private school compared to similar students in traditional public schools. This negative 
finding in math is consistent across all four years of outcome data examined and across most student 
subgroups. One important point of clarification is that this study focuses on students in the upper 
elementary and middle school grades, whereas nearly 50 percent of students receiving vouchers are 
actually in the lower elementary grades of kindergarten through fourth grade. While the exclusion of 
students in non-tested grades was unavoidable for their participant effects analysis, in our study we pay 
attention to all voucher users when judging the competitive effects of the program, incorporating 
information on all voucher-accepting private schools. 

Beyond student test scores, parent survey data from Catt and Rhinesmith (2017) offer further 
insight into Indiana’s targeted voucher program. The majority of ICSP participants who responded to the 
survey are somewhat or completely satisfied with the program, generally (86 percent), and their child’s 
school, specifically (81 percent). Compared to a child’s previous school, ICSP families in the sample 
communicate more frequently with teachers at their child’s new school (61 percent), participate in school 
activities at a higher rate (61 percent), spend more time working on math or arithmetic with their students 
(55 percent), and volunteer or perform more community service than before (55 percent). 

Survey findings are also useful in aiding our understanding of the program as they shed light on 
how much time Indiana students currently travel to attend school and how much time families would be 
willing to have their child travel to attend a different school that might be a better fit. This information 
can offer insight on any issues of school supply, private school quality, and the competitive threat to 
public schools associated with private school proximity. The children of ICSP families in this survey 
sample are more likely than traditional public school students to attend a school that is located more than 
10 minutes away from home, indicating a willingness to take on a longer commute in order to reach a 
desirable school (EdChoice, 2017). ICSP families are also more likely to say they are willing to have their 
child travel more than 30 minutes to attend a school that might be a better fit. Thus, it seems reasonable to 
infer from these findings that as more voucher-accepting private schools establish in closer proximity to 


traditional public schools, more families will consider sector-switching by way of a choice scholarship.° 


3 From a methodological perspective, if Indiana experienced a sudden increase in the entry of private schools after 
the establishment of the ICSP, a dynamic ex-ante distance measure could introduce selection bias into the model. To 
account for this, we rely on a priori distance measures, as explained in our methodology section. 
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The present study broadens our understanding of the general effects of Indiana’s voucher 
program by focusing on traditional public school students who might be indirectly affected by the 


program. 


Prior Literature on the Indirect Impacts of Private 


School Choice 


There have been 21 evaluations of the effect of increased competition resulting from private 
school vouchers in the United States to date (Egalite, 2013). Additionally, there have been two studies of 
the competitive effects of Florida’s tax-credit scholarship program (Figlio & Hart, 2014; Figlio, Hart, & 
Karbownik, 2020), which is also a private school choice program but with a funding stream that’s 
structured a little differently. A 2019 meta-analysis on the competitive effects of charter school, voucher 
school, and public school choice policies concludes the overall effect of competition on student 
achievement is positive (Jabbar, Fong, Germain, Li, Sanchez, Sun, & Devall, 2019). For the 17 studies 
with schools as the unit of analysis, the coefficient on private school competition from voucher or other 
forms of private school choice programs is .22 standard deviations, significant at p < .05. For studies with 
students (k= 17) or districts (k =1) as the unit of analysis, the researchers report null effects. This meta- 
analysis finds no overall negative impact of school choice on the test scores of students who are “left 
behind.” 

Prior research on the competitive effects of private school choice has focused on test score 
outcomes. Table 2 organizes this literature by methodological approach: difference-in-differences (7 
studies), fixed effects (7 studies), regression discontinuity design (5 studies), ordinary least squares 
regression (5 studies), and hierarchical linear modeling (1 study).* The difference-in-differences designs 
mirror the approach pursued here. The first difference is between schools facing differing levels of 
competitive pressure, the second difference is time. 

In addition to the variation in identification strategy just described, these studies also differ in 
how they operationalize the measure of a “competitive threat.” For example, Greene and Winters (2004), 
Greene (2001), and Chakrabarti (2013) use a binary measure indicating the receipt of an F grade to 
identify which Florida public schools experience a competitive threat because students in those traditional 
public schools suddenly became eligible to transfer to a private school at state expense. Other studies 


measure competitive pressures by proximity. These include the distance to the nearest private school, for 


4 The study counts documented here may appear greater than what is shown in Table 2 due to some studies 
employing multiple research methods. 
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example, or the density and diversity of private school competitors within a given radius (Carnoy et al., 
2007; Egalite & Mills, 2019; Figlio & Hart, 2014; Greene & Marsh, 2009; Greene & Winters, 2007; 
Greene & Winters, 2011; Jacob, 2014). 

Table 2 summarizes the overarching takeaways from this literature. Fourteen studies found the 
overall impact of competition on outcomes for students who remain in public schools was positive; seven 
studies found the impact was neutral-to-positive, meaning competitive pressure resulted in zero losses 
overall and gains for at least some students in some subject areas; and one study found the impact of 
increased competitive pressure from voucher programs on traditional public school performance was 
neutral. Interestingly, three studies report that traditional public school performance was strongest when 
there was a dramatic increase in the intensity of the competition from voucher programs (Forster, 2008a; 
Carnoy et al, 2007; Gray, Merrifield, & Adzima, 2014). This suggests that increased competitive pressure 
may result in higher test scores in traditional public schools up to some unknown point, but the specific 
nature of this relationship (e.g., if it takes a linear, quadratic, or cubic form) and the tipping point at which 


positive effects start to level off or change direction is unknown. 
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Table 2. Prior Studies of the Competitive Effects of Vouchers 


it Study Location Competition Measure Identification Strategy Study Period Test Score 
Effects 
1 Figlio, Hart, & Florida Distance, Density, Diversity, Slots, | Grade, year, and student- 2002-03 to 2016-17 Positive 
Karbownik (Tax-Credit Churches; (all five are also school fixed effects 
(2020) Scholarship) summarized in a Competitive 
Pressure Index) 
2 Figlio & Hart Florida Distance, Density, Diversity, Slots, | Difference-in-differences 1998-99 to 2006-07 Positive 
(2014) (Tax-Credit Churches 
Scholarship) 
3 Greene & Winters Florida Receipt of an 'F' grade Difference-in-differences 2001-02 to 2002-03 Positive 
(2004) (Voucher) 
4 Greene (2001) Florida Receipt of an 'F' grade Difference-in-differences 1998-99 to 1999-2000 __— Positive 
(Voucher) 
5 Chakrabarti Milwaukee, WI Share of poor children who would Difference-in-differences 1986-87 to 2001-02 Neutral 
(2008) (Voucher) qualify for vouchers to 
Positive 
6 Carnoy et al. Milwaukee, WI Share of poor children who would Difference-in-differences 1996-97 to 2004-05 Positive 
(2007) (Voucher) qualify for vouchers; Density 
7 Hoxby (2003) Milwaukee, WI Share of poor children who would Difference-in-differences 1996-97 to 1999-2000 __— Positive 
(Voucher) qualify for vouchers 
8  Figlio & Rouse Florida Receipt of an 'F' grade Difference-in-differences 1998-99 to 1999-00 Positive 
(2006) (Voucher) with school and year fixed (accountability 
effects stigma and 
vouchers, 
combined 
9 Greene & Winters Florida Density (voucher-accepting private | Grade, year, and student- 2000-01 to 2004-05 Positive 
(2011) (Disability Voucher) schools within 5 & 10 miles) school fixed effects 
10 Greene & Marsh Milwaukee, WI Density (relevant private schools Student fixed effects 1999-2000 to 2006-07 _— Positive 
(2009) (Voucher) within five different radii) 
11 Jacob (2014) Indiana Distance, Density, Diversity, School fixed effects 2008-09 to 2011-12 Neutral 
(Voucher) Concentration to 
Positive 
12. Carr Ohio Public school is designated as School fixed effects 2002-03 to 2007-08 Positive 
(2011) (Voucher) underperforming 


14 


13. Egalite and Mills = Louisiana Distance, Density, Diversity, School fixed effects; 2010-11 to 2012-13 Neutral 
(2019) (Voucher) Concentration; Receipt of a High Regression Discontinuity to 
“C” grade Design Positive 
14. Chakrabarti Florida Receipt of an 'F' grade Regression Discontinuity 1997—98 to 2001-02 Positive 
(2013) (Voucher) Design 
15 Rouse et al. Florida Receipt of an 'F' grade Regression Discontinuity 2002-03 to 2004—05 Positive 
(2013) (Voucher) Design (accountability 
stigma and 
vouchers, 
combined) 
16 West & Peterson Florida Receipt of an 'F' grade Regression Discontinuity 2001-02 to 2003-04 Positive 
(2006) (Voucher) Design 
17. Figlio & Ohio Compared public schools on either Regression Discontinuity 2005-06 to Neutral 
Karbownik (2016) (Voucher) side of the margin of becoming Design 2008-09 to 
voucher eligible Positive 
18 Forster (2008) Florida Receipt of an 'F' grade Regression 2001-02 to 2006-07 Positive 
(Voucher) 
19 Greene & Forster Milwaukee, WI Share of poor children who would Regression 1996-97 to 2000-01 Neutral 
(2002) (Voucher) qualify for vouchers to 
Positive 
20 Greene & Winters District of Columbia Distance and Density (participating Regression 2003-04 to 2004—05 Neutral 
(2007) (Voucher) private schools within | mile) 
21 Forster (2008) Ohio Public school is designated as Regression 2005-06 to 2006-07 Neutral 
(Voucher) chronically underperforming to 
Positive 
22 Greene & Forster San Antonio, TX Compared Edgewood district to Regression 1997-98 to 2000-01 Positive 
(2002) (Voucher) other districts with no voucher 
program 
23 Gray, Merrifield, San Antonio, TX Compared Edgewood district to Two-level hierarchical linear 1993-94 to 2007-08 Neutral 
& Adzima (2014) (Voucher) other districts with no voucher model to 
program Positive 


Note: Full references for each study cited in this table are provided in the References section. 
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We end this section with a discussion of the outstanding questions that have yet to be addressed 
by the competitive effects literature. First, in what ways are outcomes other than math and reading test 
scores impacted by such programs? Second, what explains the neutral to positive test score impacts across 
these various studies? Further research of additional school choice programs in other states is needed in 
order to understand if these differences are related to specific features of program design, methodological 
differences in how “competition” is operationalized, or local context. 

For example, programmatic differences include opportunities and incentives for collaboration and 
cooperation across school sectors. Differences in local context include variations in local education 
policies and labor market conditions. Perhaps competitive effects are strongest in states where the 
workforce is young and highly-educated so that public school principals are easily able to attract effective 
new teachers. Perhaps the strength of teacher unions plays a role by influencing how nimble school 
administrators can be in devising a response to attract and keep students enrolled in their school (Cowen 
& Strunk, 2015; Strunk & Grissom, 2010). Perhaps effects are strongest where public schools’ starting 
performance is lowest and there is much room for growth. There may also be locales where private school 
choice is welcomed by the public schools, instead of being viewed as a threat, as it is seen as a release 
valve for students who cannot be well served in their assigned school because of overcrowding or 
insufficient resources. 

Third, what other methodologies are available to study this phenomenon? We note that the bulk 
of prior studies on this subject have operationalized competition by using a geocoded measure such as the 
crow’s flight distance between a public and voucher-accepting private school. Rather than relying upon 
this Euclidean-distance measure, we use the actual drive-time between two points, accounting for road 
lengths, intersection turn times, speed limits, and so on. These data were collected by millions of 
anonymous cell phones and vehicle sensors and allow us to calculate precise travel time in minutes. We 
argue drive-time represents an improvement over the crows-flight distance between two points because it 
accounts for all manner of topographical, traffic, and social differences that can make the perceived 
distance between two points feel larger or smaller than the straight-line distance expressed in miles might 
otherwise communicate. 

Fourth, might variation in local education policies play a role in influencing competitive impacts? 
It could be the case that competitive effects are strongest when choice is less constrained, such as when 
charter school caps are lifted or limits on the size of a voucher program are removed. Only by drawing on 
data from multiple states can researchers begin to identify patterns in the variation in findings. Data from 
an additional state beyond those listed in Table 2 would be an improvement. Given that Indiana’s 


program is the nation’s largest voucher program, a competitive effects analysis of the ICSP provides a 
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great opportunity to shed light on the systemic effects of a school choice program in a previously 
unstudied state. 

A final outstanding question that has not yet been addressed by the competitive effects literature 
relates to the concept of a tipping point, which we define as the level at which the penetration of private 
schools reaches sufficient saturation to prompt a public school response. We argue that before this point is 
reached, public schools can more easily dismiss the private school transfer option as a legitimate threat to 
their anticipated student enrollment, squashing any efforts to enact meaningful changes in response. To 
explore this possibility, we test a variety of thresholds to explore whether or not a tipping point exists in 


Indiana’s context. 


Research Methodology 
Research Questions 


We address the following three research questions: 
1. Does increased private school competition predict changes in student achievement and graduation 
at a traditional public school? 
2. Are there heterogeneous impacts by student and school subgroups? 
3. Is there a tipping point at which increases in student transfers by way of the ICSP predict changes 


in student achievement at nearby traditional public schools? 


Data 


The data for this project come from two sources. First, student-level data on traditional public 
school students’ graduation status and math and English Language Arts (ELA) performance on the state 
test—the ISTEP+—-were provided by the Indiana Department of Education and cover school years 2010— 
11 to 2015-16. The ISTEP+ is administered annually in grades 3 through 8 and used for school and 
student accountability purposes. Test scores are standardized within grade and year to have a mean of 
zero and standard deviation of one. Physical addresses for all Indiana public and ICSP schools were also 
provided by the Indiana Department of Education. Using the Maptitude software package, we calculate 
the expected drive-time from each public school to every voucher program-participating private school. 
These data then allow us to subsequently calculate drive-time to the closest private school or the count of 
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private schools within a given drive-time cap. This measure incorporates road lengths, intersection turn 
times, and speed limits to calculate precise travel time in minutes. For purposes of our analyses, we 
assume average driving speeds for all road types.° We also limit travel times to no more than 90 minutes, 
since an estimated 98 percent of the state’s population has a work commute of less than 90 minutes (U.S. 
Census Bureau, 2015). Furthermore, prior school choice survey research notes that fewer than three 
percent of parents utilizing any form of public or private school choice are willing to commute for more 


than one hour (Catt & Rhinesmith, 2017). 


Drive-time Measure 


Distance to school is consistently shown to play an important role in families’ school decisions 
(Harris & Larsen, 2017). The majority of previous school choice competitive effect studies use some 
measure of physical distance to school to account for this fact, but distance is typically operationalized in 
a simplistic way as the Euclidean distance between two points (e.g., Egalite & Mills, 2019; Figlio & Hart, 
2014; Misra, Grimes, & Rogers, 2012), which ignores the psychological aspect of the distance between 
two points. That is, Euclidean distance may not accurately reflect a parent’s perception of how near or far 
a school actually is from home, especially where there is notable variation in traffic patterns across 
regions that affects the total time spent in a car when transitioning between school and home. If this is 
true of parents, it likely is also true of schools. Therefore, we use a measure of competitive pressure based 
on actual drive-time between schools, which we argue is superior to the alternative available measures 


because it presents a more accurate reflection of the relative distance between schools. 


Empirical Approach 


We adopt a two-part empirical approach to address our three research questions, first testing for a 
competitive impact in the program’s initial implementation year, then testing in all post-policy years. The 
difference-in-differences approach is powerful in this setting as it does not require random assignment for 
causal inference, just the implementation of a school choice policy that differentially impacts different 


groups of schools. Finally, the reader should note that charter schools are excluded from the analysis to 


5 We were unable to specify a departure time in Maptitude that would approximate times of day students travel to or 
from school; the drive-time tool uses historical traffic conditions (calculated using billions of anonymous 
measurements from cell phones and vehicle sensors) to determine the speeds used. 
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allow us to more cleanly estimate the competitive pressure experienced by district-operated public 


schools. 


Testing for a Competitive Impact in the First Year of the ICSP 

In the first part of our analysis, a school fixed effects model is employed in a difference-in- 
differences framework to estimate the effect of private school competition on public school performance 
in the program’s first year of operation, similar to the model estimated by Figlio and Hart (2014). The 
first difference is between schools facing differing levels of competitive pressure, the second difference is 
time. The intuition behind the model is that schools with low levels of competitive pressure from private 
schools form a useful counterfactual for schools with high levels of competition, after accounting for 


fixed differences between the schools and common time effects. 
Yise = Bi€s * Pp + BoXit + B3Ss¢ + As + BrVe + Eise [1] 


Ys: 18s the average standardized math or reading score for student 7 in school s, in year ¢; ds is a school 
fixed effect; C,is the measure of pre-policy competitive pressure facing school s (i.e., the drive-time to 
the nearest private school or the count of ICSP-participating private schools within a drive-time based 
radius). Because C is measured a priori, its value is constant within schools and therefore it does not need 
to be entered into the model separately as a standalone variable. P, is an indicator variable identifying the 
post-policy year, 2011-12; X; is a vector of student characteristics (gender, race, and indicators for 
free/reduced price lunch eligibility, limited English proficiency, and special education); S, is a vector of 
time-varying school characteristics (annual enrollment, shares of students of each race/ethnicity, the share 
of students eligible for free/reduced price lunch; principal gender and race/ ethnicity; teacher gender, 
race/ethnicity, and an indicator for the share of teachers who are first year teachers)°; y is a year fixed 
effect, and € is an idiosyncratic disturbance term. The B; coefficient on the two-way-interaction of the 
competition measure and the post-policy indicator is the parameter of interest. Robust standard errors are 


clustered at the school level. The key identifying assumption underlying this research design is that public 


® A case could be made for excluding principal gender and principal race as control variables from these models as 
these factors are under the control of the traditional public school and could be endogenous if affected by the 
competitive threat, such as a school that previously prioritized hiring principals of a certain race and/or gender now 
selecting the most promising candidate, irrespective of race and/or gender. The same logic could be true of the share 
of teachers who are first year teachers. We test this theory by running the models with and without these control 
variables and find little change to the results. 
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schools with greater or lesser competitive pressure would have trended similarly in the absence of state- 
funded tuition vouchers. 

As a follow-up to this analysis, we also generate a version of the drive-time variable that only 
counts drive-time to a private school that meets an enrollment threshold (e.g., drive-time to nearest private 


school that has accepted 10+ choice students). Those results are presented separately. 


Testing for a Competitive Impact in all Post-Policy Years 


It may be the case that first year impacts were muted because of the program’s small size at that 
time or because traditional public schools assumed the state supreme court would rule against the 
ambitious voucher program, which was the first program of its type to target middle-income families for 
vouchers. Fortunately, because we have data from 2010-11 through 2015-16, we are not restricted to 
looking for impacts in the first year alone. Multi-year results will be presented separately. Using this 
larger sample also allows us to estimate a dynamic model that accommodates the possibility of time- 


varying treatment effects. To accomplish this, we change the model specification in the following way: 


Yist = pyar rls * Per + BoXit + B3Sst + As + Ve + Eist [2] 


t allows the policy to have separate effects in the immediate post-policy year (2011-12), one year after 
initial adoption (2012-13), and two or more years after adoption (2013-14, 2014-15, or 2015-16). 

We argue our empirical approach avoids endogeneity bias in two primary ways. First, we ensure 
we are focusing on private schools that made location decisions many years ago by mapping private 
school location from the pre-policy period. Second, our analysis focuses on a period of dramatic and 
sudden expansion in student participation in the voucher program. This means it is unlikely that private 
schools had time to respond to the program by establishing or moving to a new location in such a short 


time frame. 


Mapping Procedure 


We use precise drive-time measures to plot the distance between a traditional public school and 
the nearest private school competitor enrolling ICSP students, accounting for variables such as traffic 
patterns, road types, and intersection turn times. 

After an initial attempt to geolocate all 2,178 Indiana public schools and voucher-accepting 


private schools (henceforth, ICSP schools) using the software, Maptitude, 346 addresses were modified in 
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order to be recognized by the software. This involved replacing P.O. box addresses with physical 
addresses, changing “State Rd.” to “SR,” changing “Co. Road” to “CR,” and so on. There were ten 
addresses the software still failed to recognized that were replaced by latitude and longitude coordinates 
identified from Google Maps. Thus, we successfully geolocated 100 percent of the 1,697 traditional 
public schools and the 342 ICSP schools in our sample. We then created distance and travel time matrices 
between all traditional public schools and ICSP schools for each school year under study. 

For visualization purposes, not analytic, we start by creating drive-time rings in five increments 
of 10 minutes from each ICSP school for 201 1—12 and 2012-13 (Figure 1). Although the rings in Figure 1 
are not used in the analysis—which instead relies on the continuous drive-time measure for more 
precision—they permit identification of geographic areas in Indiana where there are traditional public 
schools that do not have any participating private schools located within 30 minutes of their location. 
When comparing the two figures, it is also easy to spot those geographic areas experiencing change over 
time. For example, there are areas where the traditional public schools did not have any close ICSP 


schools in 201 1—12 but then did have at least one of the 58 additional ICSP schools in 2012-13. 
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Figure 1. Drive-Time Rings from Indiana Voucher Schools to Traditional Public Schools, 2011-12 and 2012-13 
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Sample Description 


The analysis sample contains 2,420,574 observations of 837,797 unique students from 2010-11 to 
2015-16 (Table 3). Approximately three-quarters (74 percent) of students are white, 10 percent are Black, 
and 10 percent are Hispanic. Almost half (47 percent) are low-income, 16 percent of students are 
designated as having special educational needs, and 6 percent have limited English proficiency. 

There are 1,403 unique traditional public schools in our sample, with an average enrollment of 
503 students. Over half of schools (55 percent) have a male principal and 91 percent of all principals are 
white, compared to just seven percent Black and one percent Hispanic. Teacher demographic 
characteristics largely mirror those of principals, with the average value of the teacher percent white 
variable at 94 percent, teacher percent black at just 4 percent, and teacher percent Hispanic at | percent. 
On average, 5 percent of teachers are in their first year of teaching. Charter schools are excluded from the 
analysis as they are not the population of interest for this particular study. 

Table 3 also presents descriptive statistics of the competition measures on which we rely. The 
average drive-time to the nearest private school competitor is 8.54 mins, the average count of private 
school competitors within 90 minutes is 104 schools, the average count of private school competitors 
within 30 minutes is 23 schools. Finally, the competitor count within 20 and 10 minutes is 11 and three 
schools, respectively. Histograms of the drive-time and competitor count measures are presented in 


Figures 2 and 3. 
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Table 3. Descriptive Characteristics of Analysis Sample, Math and English Language Arts 


Outcomes 


Student Characteristics 


Female 

White 

Black 

Hispanic 

Other Race 

Low-Income 

Special Education 

Limited English Proficiency 


Traditional Public School Characteristics 


Enrollment 

Percent White 

Percent Black 

Percent Hispanic 

Percent Other Race 
Percent Low-Income 
Principal Male 

Principal White 

Principal Black 

Principal Hispanic 
Principal Other Race 
Teacher Percent Male 
Teacher Percent White 
Teacher Percent Black 
Teacher Percent Hispanic 
Teacher Percent Other Race 


Count of Voucher Students in All Private 


Schools Within 10 Mins' Drive (2012) 


Count of Voucher Students in All Private 


Schools Within 20 Mins' Drive (2012) 


Count of Voucher Students in All Private 


Schools Within 30 Mins' Drive (2012) 


Count of Voucher Students in All Private 


Schools Within 90 Mins' Drive (2012) 


Competition Measures 


Drive-time (mins) 

Drive-time (10+ Voucher Students) 
Competitor School Count (90 mins) 
Competitor School Count (30 mins) 
Competitor School Count (20 mins) 


Mean 


0.49 
0.74 
0.10 
0.10 
0.07 
0.46 
0.16 
0.06 


503.10 
0.74 
0.10 
0.10 
0.07 
0.48 
0.55 
0.92 
0.07 
0.01 
0.01 
0.23 
0.94 
0.04 
0.01 
0.01 

33.56 


156.32 


572.59 


1309.35 


8.54 
16.43 
103.61 
22.88 
10.47 


SD 


0.50 
0.44 
0.30 
0.30 
0.25 
0.50 
0.36 
0.24 


320.33 
0.25 
0.17 
0.12 
0.05 
0.22 
0.50 
0.28 
0.25 
0.10 
0.08 
0.13 
0.13 
0.12 
0.03 
0.01 

60.51 


221.30 


401.95 


572.59 


7.61 
13.23 
39.38 
20.69 
10.83 


Min 


0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 


37.00 
0.00 
0.00 
0.00 
0.00 
0.01 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 


0.00 


0.00 


94.00 


0.02 
0.02 
15.00 
0.00 
0.00 


Max 


1.00 
1.00 
1.00 
1.00 
1.00 
1.00 
1.00 
1.00 


1,786.00 
1.00 
0.98 
0.81 
0.47 
1.00 
1.00 
1.00 
1.00 
1.00 
1.00 
0.67 
1.00 
1.00 
0.57 
0.17 

416.00 


1186.00 
1353.00 


2565.00 


42.35 
77.89 
176.00 
77.00 
53.00 
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Competitor School Count (10 mins) 2.46 2.90 0.00 16.00 


Notes: n= 2,420,574 observations, 837,797 unique students, 1,403 unique schools; ‘Competitor school count’ 
refers to the number of participating private schools within 90 mins’ drive of a given public school. 


Figure 2. Histogram of Drive-Time Measure 
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Figure 3. Histograms of the Competitor Count Measures 
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Results 
Findings from Analysis of Test Score Impacts 


We first present the findings of the impact of competition on test scores of students in traditional 
public schools, then discuss the estimates of the effects of competition on graduation rates. Table 4 
presents estimates of the effects of private school competition on public school students’ math and 
English Language Arts achievement in the first year (Columns 1, 3) and in all post-policy years (Columns 
2, 4) for the full sample of students. For the drive-time model results presented in the first row, we report 
null effects in math and a very small positive effect in English Language Arts, significant at p < .10. 
Across multiple model specifications, we never uncover a negative effect on math or English Language 


Arts achievement. 
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Table 4. Effects of Private School Competition on Public School Achievement, Math and ELA 


(1) (2) (3) (4) 
Subject Math ELA 
Outcome Year First Post- All Post First Post- All Post 
Policy Year Policy Policy Year Policy Years 
Years 
Competition Measure 
A: Drive-time 
Drive-time -0.0001 0.0007 0.0006 -0.0003 
(0.0004) (0.0005) (0.0004) (0.0004) 
B: Private School Density 
Competitor School Count (90 mins) 0.0002** 0.0003** 0.0003*** 0.0001 
(0.0001) (0.0001) (0.0001) (0.0001) 
Competitor School Count (30 mins) 0.0002 0.0002 0.0002 -0.0000 
(0.0002) (0.0002) (0.0001) (0.0002) 
Competitor School Count (20 mins) 0.0004 0.0003 0.0002 -0.0001 
(0.0003) (0.0004) (0.0003) (0.0003) 
Competitor School Count (10 mins) 0.0007 0.0010 0.0017* 0.0006 
(0.0012) (0.0016) (0.0010) (0.0011) 
R-squared 0.2177 0.2177 0.2427 0.2427 


Notes: These coefficients represent the interaction between the measure of competition and the post-policy years: 
school years ending in 2012 (the first post-policy year), 2013, 2014, 2015, and 2016. To ease interpretation, drive- 
time has been reverse-coded. ‘Competitor school count’ refers to the number of participating private schools within 
90 mins' drive of a given public school. Models include controls for pre-policy competitive pressure, student 
characteristics (gender, race, and indicators for free/reduced price lunch eligibility, limited English proficiency, and 
special education), a school fixed effect, time-varying school characteristics (annual enrollment, shares of students 
of each race/ethnicity, the share of students eligible for free/reduced price lunch and an indicator if that share 
exceeds 40 percent after 2014-15, triggering the Community Eligibility Provision; principal gender and race/ 
ethnicity; teacher gender, race/ethnicity, and an indicator for the share of teachers who are first year teachers), and a 
year fixed effect. Robust standard errors clustered by school in parentheses. N = 2,420,574. *** p<0.01, ** p<0.05, 
* p<0.10 


In what follows, we discuss the findings associated with each of the competition measures 


employed. We also discuss the findings associated with the dosage analysis. 


Drive-time to Nearest Private School as a Measure of Competition 

A decrease in the drive-time to the nearest private school competitor is associated with no change 
in public school math achievement in the years after the policy went into effect. Looking at English 
Language Arts achievement, a ten-minute decrease in the drive-time to the nearest private school 


competitor is associated with a 0.007 standard deviation (SD) increase in public school student 
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achievement in the year after the policy went into effect.’ Although the finding is statistically significant, 
this is a very small effect. A ten-minute decrease in drive-time is associated with an increase in English 
Language Arts achievement of less than one-percent of a standard deviation, which is smaller than a one- 
percentile change. To make sure this finding isn’t being driven by our choice of competition measure, we 
also use the drive-time data to generate alternative measures of competition, including a private school 
density measure we call the “Competitor School Count.” This represents the count of private schools 
within a given drive-time radius of a public school. Findings from these models are consistent with the 


drive-time results already discussed. 


Competitor School Count as a Measure of Competition 


Examining public school responses to the competitor school count in 90-, 30-, 20-, and 10- 
minute radii, we find essentially null effects. That is, although some specifications reveal a statistically 
significant, positive effect, it is too small to rule out a spurious relationship. For example, an increase of 
one private school competitor within a 90-minute radius of a given public school is associated with a 
0.0002 SD increase in math achievement and a 0.0003 SD increase in English Language Arts 
achievement in the first year after the policy went into effect. Examining impacts across all post-policy 
years, the math effect remains statistically significant at 0.0003 SD whereas the English Language Arts 
effect is essentially zero. In sum, increased private school competition is associated with an increase in 
student test scores of zero to approximately one percent of a standard deviation, depending on the 


particular measure used. 


Results of Tipping Point Analysis 

It might be the case, however, that only private schools receiving a critical mass of voucher 
students actually represent a tangible competitive “threat” to the traditional public schools in that area. To 
test this dosage theory, we check if there is a change in public school students’ achievement in the first 
post-policy year that is associated with a reduction in the drive-time distance to the nearest private school 
that has accepted a critical mass of voucher students—a minimum of 10, 20, 30, 40, or 50 voucher 
students (Table 5). At the lowest threshold of 10 voucher students, we observe a statistically significant 
positive effect on English Language Arts achievement. Specifically, a ten-minute decrease in the drive- 


time to a private school with at least ten voucher students is associated with an increase in public school 


’ To ease interpretation, we multiply coefficients by ten to get the change in achievement associated with a ten- 
minute decrease in drive-time. 
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achievement of 0.006 SD in English Language Arts (i.e., approximately half of one percent of a standard 
deviation), which is a very small but statistically significant positive effect. The effect at higher 
concentrations of voucher students remains similar in both magnitude and direction, suggesting there is 
not a strong dosage effect at play. Thus, it appears that the private school threat elicits a small positive 
change in public school achievement after the voucher program is enacted but the public school response 


is not mediated by the perceived size of the competitive threat, at least in terms of voucher student count. 


Table 5. Dosage Effects of Private School Competition on TPS Achievement, Math and ELA 


(1) (2) 
Subject Math ELA 
Drive-time to Private School with 10+ Voucher Students 0.0001 0.0006*** 
(0.0003) (0.0002) 
Drive-time to Private School with 20+ Voucher Students -0.0002 0.0001 
(0.0002) (0.0001) 
Drive-time to Private School with 30+ Voucher Students -0.0000 0.0001 
(0.0002) (0.0001) 
Drive-time to Private School with 40+ Voucher Students 0.0002 0.0001 
(0.0002) (0.0001) 
Drive-time to Private School with 50+ Voucher Students 0.0002 0.0001 
(0.0002) (0.0001) 
R-squared 0.2175 0.2424 


Notes: These coefficients represent the interaction between the measure of competition and the first post-policy 
year: school year ending in 2012. Drive-time is reverse-coded for ease of interpretation. To ease interpretation, 
coefficients can be multiplied by 10 to represent the achievement change associated with a 10-minute decrease in 
drive-time. Models include controls for pre-policy competitive pressure, student characteristics (gender, race, and 
indicators for free/reduced price lunch eligibility, limited English proficiency, and special education), a school fixed 
effect, time-varying school characteristics (annual enrollment, shares of students of each race/ethnicity, the share of 
students eligible for free/reduced price lunch and an indicator if that share exceeds 40 percent after 2014-15, 
triggering the Community Eligibility Provision; principal gender and race/ ethnicity; teacher gender, race/ethnicity, 
and an indicator for the share of teachers who are first year teachers), and a year fixed effect. Standard errors 
clustered by school in parentheses. n = 2,385,571 


When interpreting these findings, it is helpful to note the jump in student participation in Year 3, 
which may have represented a sea change for traditional public schools who had been casually observing 
up to that point how eligible families were responding to the voucher option. Specifically, student 
enrollment more than doubled from Year | to Year 2, increasing from 3,911 to 9,139 students and then 
jumped to 19,809 students by Year 3. Given the dramatic increase in student participation over time due 


to the expansion and then removal of participation caps, we might expect to observe different effects for 
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each of the outcome years examined in this analysis. In particular, Year 3 may be an inflection point at 
which it became clear to traditional public schools that the voucher program was a legitimate threat that 
was now beginning to grow at a substantially faster rate. Also relevant context is that the Republican 
candidate for governor and a supporter of the state’s voucher program, Congressman Mike Pence, won 
the gubernatorial election in November 2012, signaling to voucher opponents that political support for the 
school choice program would continue. Finally, it may also be the case that traditional public schools 
needed multiple years to respond to the voucher “threat” and the initial improvements that had taken root 
in earlier years took until Year 3 to bear fruit. 

Figure 4 documents impact estimates for each outcome year separately. It is interesting to note 
the positive and statistically significant increase in math scores that occurred in Year 3, which is 
consistent with the hypothesis that the competitive effect became most salient in this year. In contrast, the 


effects for English Language Arts are statistically equivalent to zero in all outcome years. 
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Figure 4. Competitive Effects by Individual Outcome Years, Math and ELA, Drive-time Measure 


Impacts by Outcome Year: smath (drivetime_r) 


1 Year Effects 


2 Year Effects 


3 Year Effects <= «~~ 


4 Year Effects 


5 Year Effects 


T T T T T 
-.003 -.002 -.001 0 .001 .002 .003 
Impacts by Outcome Year: sela (drivetime_r) 


1 Year Effects 


2 Year Effects 


3 Year Effects 


4 Year Effects 


5 Year Effects 


T T T T T T 
-.003 -.002 -.001 0 .001 .002 .003 


Jl 


Figure 5 documents a similar pattern using the dosage analysis, which examines the effect of 
drive-time to the nearest private school with 10, 20, 30, 40, or 50 voucher students.® 


Figure 5. Competitive Effects by Individual Outcome Years, Math and ELA, Drive-time to Nearest 
Private School with at Least 10 Voucher Students 


Impacts by Outcome Year: smath (drivetimeto10_r) 
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8 We also repeat this analysis using the competitor count variable measured in 90, 30, 20, and 10- minute radii. The 
effects for individual outcome years are zero in the majority of cases. Any small positive or negative individual year 
effects never exceed one percent of a standard deviation. Results are available by request. 
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Subgroup Findings, Achievement Analysis 


Thus far, we have identified neutral to small positive overall responses to private school 
competition after Indiana’s voucher bill became law. We now check for subgroup effects. The public 
school student subgroups examined are female, white, black, Hispanic, low-income, special education, 
limited English proficient, and elementary-aged students (Table 6). Interpreting the model that 
incorporates all available years of data, which is our preferred specification, we observe positive math 
effects associated with the drive-time measure for low-income and special education students. 
Specifically, a ten-minute decrease in drive-time to the nearest private school competitor is associated 


with an achievement increase of 0.012 (approximately one percent of a standard deviation) for low- 


income students and 0.021 (approximately two percent of a standard deviation) for students with special 


needs, which is an interesting finding given that these are two subgroups of students specifically targeted 


by the voucher program’s eligibility rules.° Still looking at math outcomes, the count of private school 
competitors within a 90-minute drive is associated with increases in public school achievement for the 
following student subgroups: female, white, Hispanic, low-income, special-education, limited English 


proficient, and elementary-aged students. 


° The ten-minute increment is used here as it is more intuitive than a one-minute change in drive-time. Adjustments 
of this nature are commonly used to show the marginal effects of continuous variables. For reference, 10 minutes is 


less than one standard deviation of drive-time (SD = 7.63). 
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Table 6. Effects of Private School Competition on Public School Achievement, Student Subgroups 


Math, First Post-Policy Year 


Drive-time 

Competitor Sch Count (90 mins) 
Competitor Sch Count (30 mins) 
Competitor Sch Count (20 mins) 


Competitor Sch Count (10 mins) 


Math, All Post-Policy Years 


Drive-time 

Competitor Sch Count (90 mins) 
Competitor Sch Count (30 mins) 
Competitor Sch Count (20 mins) 
Competitor Sch Count (10 mins) 


ELA, First Post-Policy Year 
Drive-time 


Competitor Sch Count (90 mins) 
Competitor Sch Count (30 mins) 
Competitor Sch Count (20 mins) 


Competitor Sch Count (10 mins) 


1 2 3 4 5 6 7 8 
Female White Black Hispanic Low- Special LEP Elementary 
Income Education Grades 
-0.0002 0.0004 -0.0046 -0.0011 -0.0007 -0.0002 -0.0027* 0.0005 
(0.0004) (0.0004) (0.0032) (0.0011) (0.0005) (0.0008) (0.0015) (0.0006) 
0.0001 0.0002** -0.0001 0.0002 0.0001 0.0002 0.0000 -0.0000 
(0.0001) (0.0001) (0.0002) (0.0002) (0.0001) (0.0002) (0.0003) (0.0001) 
0.0002 0.0004** 0.0002 0.0006 0.0000 0.0004 0.0003 0.0003 
(0.0002) (0.0002) (0.0004) (0.0004) (0.0002) (0.0003) (0.0005) (0.0002) 
0.0002 0.0007** 0.0006 0.0014** 0.0002 0.0010* 0.0009 0.0009* 
(0.0003) (0.0004) (0.0008) (0.0006) (0.0004) (0.0006) (0.0008) (0.0004) 
0.0002 0.0017 -0.0005 0.0040* 0.0004 0.0030 0.0042 0.0026* 
(0.0013) (0.0014) (0.0025) (0.0023) (0.0013) (0.0022) (0.0031) (0.0016) 
0.0008 0.0001 0.0085 0.0004 0.0012** 0.0021** -0.0007 0.0003 
(0.0005) (0.0005) (0.0054) (0.0013) (0.0006) (0.0010) (0.0020) (0.0007) 
0.0003 ** 0.0002** 0.0004 0.0007*** 0.0003 ** 0.0004** 0.0007** 0.0006*** 
(0.0001) (0.0001) (0.0003) (0.0002) (0.0001) (0.0002) (0.0003) (0.0001) 
0.0002 -0.0000 0.0003 0.0002 0.0004 0.0005 -0.0003 0.0003 
(0.0002) (0.0002) (0.0005) (0.0004) (0.0003) (0.0004) (0.0006) (0.0003) 
0.0004 -0.0002 0.0007 -0.0002 0.0007 0.0011 -0.0017* -0.0001 
(0.0004) (0.0005) (0.0010) (0.0007) (0.0005) (0.0008) (0.0009) (0.0005) 
0.0013 -0.0004 0.0030 -0.0040 0.0016 0.0055* -0.0094** -0.0014 
(0.0017) (0.0017) (0.0041) (0.0026) (0.0018) (0.0030) (0.0038) (0.0019) 
0.0004 0.0011*** -0.0010 -0.0015 -0.0001 -0.0001 -0.0029* 0.0009* 
(0.0004) (0.0004) (0.0019) (0.0011) (0.0004) (0.0008) (0.0015) (0.0005) 
0.0002*** = 0.0003*** -0.0000 0.0002 0.0002** 0.0002 0.0002 0.0001 
(0.0001) (0.0001) (0.0002) (0.0002) (0.0001) (0.0002) (0.0003) (0.0001) 
0.0001 0.0006*** 0.0001 0.0002 0.0000 0.0004 -0.0003 0.0004** 
(0.0001) (0.0002) (0.0003) (0.0003) (0.0002) (0.0003) (0.0004) (0.0002) 
0.0000 0.0010*** 0.0003 0.0004 0.0000 0.0009* -0.0005 0.0007** 
(0.0003) (0.0003) (0.0005) (0.0005) (0.0003) (0.0005) (0.0008) (0.0003) 
0.0011 0.0038*** 0.0005 0.0031 0.0011 0.0037* 0.0018 0.0033 *** 


34 


ELA, All Post-Policy Years 
Drive-time 


Competitor Sch Count (90 mins) 
Competitor Sch Count (30 mins) 
Competitor Sch Count (20 mins) 
Competitor Sch Count (10 mins) 


Observations 
R-squared 


(0.0010) (0.0012) 


-0.0003 -0.0009** 
(0.0004) (0.0004) 
0.0001 -0.0000 


(0.0001) (0.0001) 
-0.0001 -0.0005*** 
(0.0002) (0.0002) 


-0.0001 -0.0006* 
(0.0003) (0.0004) 
0.0006 -0.0006 


(0.0011) (0.0014) 
1,191,407 1,779,735 
0.2258 0.2060 


(0.0018) 


0.0053 
(0.0033) 
0.0004* 
(0.0002) 
0.0001 
(0.0004) 
-0.0001 
(0.0007) 
0.0025 
(0.0029) 
236,939 
0.2104 


(0.0019) 


0.0009 
(0.0010) 
0.0005*** 
(0.0002) 
0.0004 
(0.0003) 
0.0000 
(0.0005) 
-0.0011 
(0.0019) 
241,088 
0.2396 


(0.0010) 


0.0004 
(0.0005) 
0.0002* 
(0.0001) 

0.0003 
(0.0002) 
0.0004 
(0.0003) 
0.0015 
(0.0012) 
1,142,533 
0.1896 


(0.0020) 


0.0006 
(0.0008) 
0.0000 
(0.0002) 
-0.0000 
(0.0003) 
0.0002 
(0.0006) 
0.0021 
(0.0025) 
383,648 
0.1456 


(0.0030) 


0.0012 
(0.0016) 
0.0003 
(0.0003) 
0.0002 
(0.0004) 
-0.0011 
(0.0007) 
-0.0046* 
(0.0028) 
148,886 
0.1229 


(0.0012) 


-0.0003 
(0.0005) 
0.0003** 
(0.0001) 
0.0000 
(0.0002) 
-0.0002 
(0.0004) 
-0.0010 
(0.0013) 
1,221,057 
0.2454 


Notes: These coefficients represent the interaction between the measure of competition and an indicator for the post-policy year(s). Drive-time is reverse- 
coded for ease of interpretation. Models include controls for pre-policy competitive pressure, student characteristics (gender, race, and indicators for 
free/reduced price lunch eligibility, limited English proficiency, and special education), a school fixed effect, time-varying school characteristics (annual 
enrollment, shares of students of each race/ethnicity, the share of students eligible for free/reduced price lunch and an indicator if that share exceeds 40 


percent after 2014-15, triggering the Community Eligibility Provision; principal gender and race/ ethnicity; teacher gender, race/ethnicity, and an indicator 


for the share of teachers who are first year teachers), and a year fixed effect. Robust standard errors clustered by school in parentheses. *** p<0.01, ** 


p<0.05, * p<0.10 


Turning next to the student subgroup findings for English Language Arts achievement under our preferred specification, we observe small 


negative impacts for white students of -0.001 SD under the drive-time measure (scaled for a ten-minute reduction instead of one-minute) and null 


effects for other student subgroups. Using the competitor school count with a 90 min drive-time radius, we observe positive impacts for Hispanic 


students (0.0005 SD) and elementary-aged students (0.0003 SD). 
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Table 7. Dosage Effects of Private School Competition on Public School Achievement, Student Subgroups 


1 v 3 4 5 6 7 8 
Female White Black Hispanic Low-Income Special LEP Elementary 
Education Grades 
Math 
Drive-time to 10+ 0.0007** 0.0004 0.0042** 0.0002 0.0012*** 0.0023*** -0.0007 0.0004 
(0.0003) (0.0003) (0.0020) (0.0008) (0.0004) (0.0007) (0.0010) (0.0004) 
Drive-time to 20+ 0.0007*** 0.0005** 0.0013*** 0.0007 0.0009*** 0.0017*** 0.0001 0.0009*** 
(0.0002) (0.0002) (0.0005) (0.0005) (0.0002) (0.0004) (0.0007) (0.0003) 
Drive-time to 30+ 0.0005** 0.0005** 0.0006 -0.0006 0.0006** 0.0014*** -0.0012** 0.0006** 
(0.0002) (0.0002) (0.0005) (0.0004) (0.0002) (0.0004) (0.0006) (0.0003) 
Drive-time to 40+ 0.0003 0.0001 0.0002 -0.0010** 0.0004 0.0009** -0.0016** 0.0003 
(0.0002) (0.0002) (0.0007) (0.0005) (0.0003) (0.0004) (0.0007) (0.0003) 
Drive-time to 50+ 0.0004* 0.0003 -0.0001 -0.0009* 0.0005* 0.0011** -0.0015** 0.0004 
(0.0002) (0.0002) (0.0006) (0.0004) (0.0003) (0.0004) (0.0006) (0.0003) 
Drive-time to 100+ -0.0005 -0.0005 -0.0017** -0.0008 -0.0008* -0.0006 -0.0017*** -0.0001 
ELA 
Drive-time to 10+ 0.0004 0.0001 0.0036** 0.0003 0.0009*** 0.0014*** -0.0007 0.0003 
(0.0002) (0.0003) (0.0014) (0.0006) (0.0003) (0.0006) (0.0009) (0.0003) 
Drive-time to 20+ 0.0004** 0.0001 0.0005 0.0005 0.0005*** 0.0008** 0.0002 0.0004** 
(0.0002) (0.0002) (0.0004) (0.0004) (0.0002) (0.0004) (0.0006) (0.0002) 
Drive-time to 30+ 0.0002 0.0000 -0.0002 -0.0006* 0.0003 0.0007* -0.0006 0.0002 
(0.0002) (0.0002) (0.0004) (0.0003) (0.0002) (0.0003) (0.0005) (0.0002) 
Drive-time to 40+ 0.0001 -0.0002 -0.0005 -0.0008** 0.0001 0.0003 -0.0007 0.0002 
(0.0002) (0.0002) (0.0006) (0.0004) (0.0002) (0.0004) (0.0006) (0.0002) 
Pave Hae AOU 0.0002 -0.0001 -0.0007 -0.0007** 0.0002 0.0004 -0.0007 0.0002 
(0.0002) (0.0002) (0.0005) (0.0004) (0.0002) (0.0004) (0.0006) (0.0002) 
Observations 1,174,054 1,754,107 233,207 237,600 1,124,118 377,529 146,872 1,209,532 
R-squared 0.2163 0.1770 0.1599 0.1762 0.1466 0.1452 0.1476 0.2144 


Notes: These coefficients represent the interaction between the measure of competition and the post-policy years. Drive-time is reverse-coded for ease of 
interpretation. Models include controls for pre-policy competitive pressure, student characteristics (gender, race, and indicators for free/reduced price lunch 
eligibility, limited English proficiency, and special education), a school fixed effect, time-varying school characteristics (annual enrollment, shares of 
students of each race/ethnicity, the share of students eligible for free/reduced price lunch and an indicator if that share exceeds 40 percent after 2014-15, 
triggering the Community Eligibility Provision; principal gender and race/ ethnicity; teacher gender, race/ethnicity, and an indicator for the share of teachers 
who are first year teachers), and a year fixed effect. Robust standard errors clustered by school in parentheses. *** p<0.01, ** p<0.05, * p<0.10 
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We also check for dosage effects among these student subgroups (Table 7). As before, no clear 
patterns emerge, indicating that the overall perceived threat of private school proximity is not 
differentiated by the number of voucher students attending nearby private schools in the post-policy 
years. 

Thus far, we have examined the achievement outcomes for specific student subgroups within the 
public school sample. It may be the case, however, that the more relevant subgroup is at the school level, 
not the student level. That is, the voucher program may be more likely to elicit a competitive response 
from certain types of public schools defined by the populations they serve, such as schools serving large 
proportions of low-income students or students of color, many of whom qualify for the voucher program. 
Thus, Table 8 presents the competitive effects impacts, broken out this time by the following school-level 
subgroups: high-poverty schools (defined as public schools with student poverty levels above the 75th 
percentile for the state of Indiana), low-poverty schools (defined as public schools with student poverty 
levels below the 25th percentile), high-minority schools (defined as schools in which the percent of 
students who are white is below the 25th percentile), low-minority schools (defined as schools in which 
the percent of students who are white is above the 75th percentile), and urban/suburban schools (defined 
as schools located in a locale coded as a large, mid-size, or small city or suburb, as compared to schools 
located in locales coded as a town or rural area). No clear patterns emerge from these data, leading us to 


conclude there is a null effect. 
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Table 8. Effects of Private School Competition on Public School Achievement, School Subgroups 


() (2) (3) (4) (5) 
High- Low- High- Low- Urban or 
Poverty Poverty Minority Minority Suburban 
Schools Schools Schools Schools Schools 
Math, First Post Policy Year 
Drive-time -0.0019 -0.0003 -0.0017 0.0010 -0.0006 
(0.0012) (0.0017) (0.0019) (0.0008) (0.0021) 
Competitor School Count (90 mins) 0.0000 0.0003 -0.0000 0.0001 0.0001 
(0.0002) (0.0002) (0.0002) (0.0002) (0.0001) 
Competitor School Count (30 mins) 0.0003 0.0004 0.0004 0.0004 0.0000 
(0.0003) (0.0004) (0.0004) (0.0009) (0.0003) 
Competitor School Count (20 mins) 0.0009* 0.0014 0.0013* 0.0018 0.0004 
(0.0006) (0.0012) (0.0007) (0.0027) (0.0005) 
Competitor School Count (10 mins) 0.0034* -0.0017 0.0050** -0.0012 0.0011 
(0.0019) (0.0044) (0.0024) (0.0099) (0.0015) 
Math, All Post Policy Years 
Drive-time 0.0010 -0.0022 -0.0000 -0.0001 0.0024 
(0.0018) (0.0022) (0.0030) (0.0009) (0.0026) 
Competitor School Count (90 mins) 0.0002 0.0004 0.0002 0.0002 0.0004** 
(0.0003) (0.0003) (0.0003) (0.0002) (0.0002) 
Competitor School Count (30 mins) -0.0000 -0.0003 -0.0003 0.0021* -0.0001 
(0.0004) (0.0006) (0.0004) (0.0010) (0.0003) 
Competitor School Count (20 mins) -0.0000 -0.0028* -0.0006 0.0041 -0.0003 
(0.0007) (0.0014) (0.0008) (0.0031) (0.0006) 
Competitor School Count (10 mins) -0.0003 -0.0047 -0.0022 -0.0200* -0.0012 
(0.0029) (0.0052) (0.0031) (0.0108) (0.0021) 
ELA, First Post Policy Year 
Drive-time -0.0019 0.0033** -0.0014 0.0011 0.0019 
(0.0012) (0.0015) (0.0021) (0.0007) (0.0017) 
Competitor School Count (90 mins) 0.0002 0.0005*** 0.0000 0.0003* 0.0001 
(0.0001) (0.0002) (0.0002) (0.0001) (0.0001) 
Competitor School Count (30 mins) 0.0000 0.0003 0.0001 0.0011 -0.0002 
(0.0002) (0.0004) (0.0003) (0.0009) (0.0002) 
Competitor School Count (20 mins) 0.0001 0.0001 0.0004 0.0028 -0.0005 
(0.0004) (0.0011) (0.0006) (0.0024) (0.0003) 
Competitor School Count (10 mins) 0.0017 0.0022 0.0041** 0.0014 0.0002 
(0.0015) (0.0046) (0.0020) (0.0084) (0.0013) 
ELA, All Post Policy Years 
Drive-time 0.0016 -0.0018 0.0020 -0.0009 0.0011 
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(0.0014) (0.0017) (0.0019) (0.0006) (0.0018) 


Competitor School Count (90 mins) 0.0003 -0.0002 0.0003 -0.0000 0.0002* 
(0.0002) (0.0002) (0.0002) (0.0001) (0.0001) 
Competitor School Count (30 mins) 0.0004 —--0.001 1 0.0001 -0.0008 -0.0000 
(0.0003) (0.0004) (0.0003) (0.0007) (0.0002) 
Competitor School Count (20 mins) 0.0002 —--0.0031*** —— -0,0004 -0.0024 -0.0001 
(0.0005) (0.0011) (0.0006) (0.0022) (0.0005) 
Competitor School Count (10 mins) 0.0017 -0.0066 -0.0011 -0.0158* 0.0005 


(0.0021) (0.0041) (0.0024) (0.0092) (0.0015) 


R-squared 0.2063 0.1833 0.2404 0.1896 0.2665 


Notes: These coefficients represent the interaction between the measure of competition and the post-policy year(s). 
Drive-time is reverse-coded for ease of interpretation. Models include controls for pre-policy competitive pressure, 
student characteristics (gender, race, and indicators for free/reduced price lunch eligibility, limited English 
proficiency, and special education), a school fixed effect, time-varying school characteristics (annual enrollment, 
shares of students of each race/ethnicity, the share of students eligible for free/reduced price lunch; principal gender 
and race/ ethnicity; teacher gender, race/ethnicity, and an indicator for the share of teachers who are first year 
teachers), and a year fixed effect. Robust standard errors clustered by school in parentheses. High poverty schools 
are defined as schools with poverty levels above the 75th percentile; low poverty schools are defined as schools with 
poverty levels below the 25th percentile. High-minority schools are defined as schools in which the percent of 
students who are white is below the 25th percentile; low-minority schools are defined as schools in which the 
percent of students who are white is above the 75th percentile. The 'Urban or Suburban' subgroup refers to schools 
located in a locale coded as a large, mid-size, or small city or suburb, as compared to schools located in locales 
coded as a town or rural area. n = 1,264,384 *** p<0.01, ** p<0.05, * p<0.10 


We also check for dosage effects among these school-level subgroups (Table 9) but there is little 


evidence of a pattern of impacts that corresponds with the dosage hypothesis. 


Table 9. Dosage Effects of Private School Competition on Public School Achievement, School 


Subgroups 
() (2) (3) (4) (5) 
High- Low- High- Low- Urban or 
Poverty Poverty Minority Minority Suburban 
Schools Schools Schools Schools Schools 
Math 
Drive-time to Private School 0.0010 -0.0005 -0.0005 0.0012* 0.0023** 
With 10+ (0.0011) (0.0011) (0.0019) (0.0006) (0.0010) 
Drive-time to Private School 0.0006 0.0005 0.0009 0.0007*  0.0012*** 
Wir (0.0005) (0.0008) (0.0008) (0.0004) (0.0003) 
Drive-time to Private School 0.0000 0.0004 -0.0008 0.0006 0.0007** 
Wi (0.0005) (0.0007) (0.0007) (0.0004) (0.0003) 
Drive-time to Private School -0.0001 -0.0002 —_-0.0018*** — 0.0009* -0.0000 
With 40+ 


(0.0005) (0.0008) (0.0006) ~—- (0.0005) ~— (0.0004) 
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Drive-time to Private School 0.0001 0.0005 — -0.0018*** — 0.0010** 0.0001 


a 
With 50 (0.0005) (0.0007) (0.0005) (0.0005) ~—-(0.0004) 
ELA 
Drive-time to Private School 0.0018** -0.0009 0.0013 0.0006 0.0020** 
With: 10s (0.0008) (0.0007) (0.0014) ~— (0.0005) ~—_(0.0008) 
Drive-time to Private School 0.0009** -0.0000 0.0004 0.0001 0.0007*** 
With: 20+ (0.0004) (0.0004) (0.0005) (0.0003) (0.0003) 
Drive-time to Private School 0.0003 -0.0002 -0.0009** -0.0000 0.0003 
With 30+ (0.0004) (0.0004) (0.0004) (0.0003) ~—- (0.0003) 
Drive-time to Private School 0.0003 -0.0002 -0.0013*** 0.0000 0.0001 
With a0 (0.0004) (0.0006) (0.0005) (0.0003) (0.0003) 
Drive-time to Private School 0.0004 -0.0002 -0.0012*** 0.0000 0.0001 
With 50+ (0.0004) (0.0006) (0.0005) (0.0003) ~—_ (0.0003) 
R-squared 0.2053 0.1833 0.2397 0.1884 0.2664 


Notes: These coefficients represent the interaction between the measure of competition and the post-policy years. 
Drive-time is reverse-coded for ease of interpretation. Models include controls for pre-policy competitive pressure, 
student characteristics (gender, race, and indicators for free/reduced price lunch eligibility, limited English 
proficiency, and special education), a school fixed effect, time-varying school characteristics (annual enrollment, 
shares of students of each race/ethnicity, the share of students eligible for free/reduced price lunch and an indicator 
if that share exceeds 40 percent after 2014-15, triggering the Community Eligibility Provision; principal gender and 
race/ ethnicity; teacher gender, race/ethnicity, and an indicator for the share of teachers who are first year teachers), 
and a year fixed effect. Robust standard errors clustered by school in parentheses. High poverty schools are defined 
as schools with poverty levels above the 75th percentile; low poverty schools are defined as schools with poverty 
levels below the 25th percentile. High-minority schools are defined as schools in which the percent of students who 
are white is below the 25th percentile; low-minority schools are defined as schools in which the percent of students 
who are white is above the 75th percentile. The 'Urban or Suburban' subgroup refers to schools located in a locale 
coded as a large, mid-size, or small city or suburb, as compared to schools located in locales coded as a town or rural 
area. n= 1,261,763 *** p<0.01, ** p<0.05, * p<0.10 


Findings from Analysis of Graduation Impacts 


We turn next to the findings of the impact analysis examining traditional public school 
graduation rates as the outcome variable. Graduation rates are calculated using the standard four- 
year adjusted cohort method. The student sample for this analysis consists of public school 
students who were expected to graduate in 2011 (i.e., one year before the voucher policy became 
law), 2012 (the first outcome year), or later. This includes 434,596 students in 369 traditional 
public schools. Descriptive statistics (Table Al of the appendix) reveal a very similar distribution 
of student characteristics as in the test score impact sample. The majority (77 percent) of students 


are white, 10 percent are Black, and seven percent are Hispanic. Thirty-nine percent are low- 
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income, 13 percent are labeled as having special educational needs, and three percent are limited 
English proficient. School-level characteristics are also similar as before, although school 
enrollment sizes are much larger now, with a mean enrollment value of 1,521 students, which is 
unsurprising as the sample for this particular analysis consists exclusively of high-school 
students. 

Student graduation status is a binary variable, coded one to indicate a student graduated 
on time (85 percent of students in our overall sample) and zero otherwise. Reasons offered for 
non-graduation include removal by parents (31 percent of non-graduates), disinterest in 
curriculum (7 percent of non-graduates), and expulsion (1 percent). Foreign exchange students 
and those who moved out of state are removed from the sample altogether. 

Table 10 displays the overall and subgroup effects of private school competition on 
traditional public school students’ probability of graduating on time. To ease interpretation, we 
use a linear probability model. Effects can be interpreted in the following manner: An increase in 
private school competition operationalized as a one-minute reduction in drive-time to the nearest 
competitor is associated with a 8B percent change in a student's probability of graduating on time 
from high school, holding everything else constant. Overall, we find null effects on students’ 
probability of graduating from high school. Subgroup findings are largely null, although there is 
some evidence that students in high-poverty traditional public schools have experienced a one 
percent increase in their probability of graduating in the first year after the voucher policy went 


into effect. However, this effect fades away in subsequent years. 
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Table 10. Effects of Private School Competition on Traditional Public School Students' Probability 


of Graduating On Time 


(1) (2) 
First Post-Policy Year All Post-Policy Years 
Main Effects 
Drive-time -0.0001 0.0004 
(0.0002) (0.0003) 
Student Subgroups 
Female -0.0002 0.0004 
(0.0003) (0.0003) 
White 0.0001 0.0006** 
(0.0003) (0.0003) 
Black 0.0007 0.0009 
(0.0025) (0.0021) 
Hispanic -0.0010 -0.0009 
(0.0010) (0.0010) 
How peome 0.0001 0.0004 
. : (0.0004) (0.0004) 
Special Education 0.0011 0.0003 
(0.0007) (0.0007) 
ne 0.0003 0.0013 
(0.0020) (0.0017) 
School Subgroups 
High Poverty Schools 0.0111** -0.0014 
(0.0041) (0.0056) 
Low Poverty Schools -0.0006 0.0010* 
(0.0006) (0.0006) 
High Minority Schools 0.0022 -0.0009 
(0.0036) (0.0021) 
Low Minority Schools 0.0005 0.0005 
(0.0004) (0.0004) 
Urban or Suburban 0.0010 0.0001 
(0.0012) (0.0021) 


Notes: These coefficients represent the interaction between the measure of competition and the post-policy years: 
school years ending in 2012 (the first post-policy year), 2013, 2014, 2015, and 2016. Drive-time is reverse-coded for 
ease of interpretation. Models include controls for pre-policy competitive pressure, student characteristics (gender, 
race, and indicators for free/reduced price lunch eligibility, limited English proficiency, and special education), a 
school fixed effect, time-varying school characteristics (annual enrollment, shares of students of each race/ethnicity, 
the share of students eligible for free/reduced price lunch and an indicator if that share exceeds 40 percent after 
2014-15, triggering the Community Eligibility Provision; principal gender and race/ ethnicity; teacher gender, 
race/ethnicity, and an indicator for the share of teachers who are first year teachers), and a year fixed effect. 
Standard errors clustered by school in parentheses. n = 434,596 *** p<0.01, ** p<0.05, * p<0.10 


42 


It is also helpful to consider the graduation impact by individual outcome year, in case 
the overall effect masks heterogeneity over time, but that does not appear to be the case in 
Indiana in this time period. Figure 10 plots the coefficient estimates for each individual outcome 
year. Graduation effects are insignificant in all five years examined, although it is interesting to 
note the graduation effect is approaching statistical significance at standard levels (specifically, p 


= .063) in Year 3, which is when the program experienced dramatic growth for the first time. 


Figure 6. Competitive Effects on Traditional Public School Students’ Likelihood of 
Graduation, Drive-time Measure 


Impacts by Outcome Year: Graduation (drivetime_r) 


1 Year Effects v 
2 Year Effects v 
3 Year Effects &. 
4 Year Effects V 
5 Year Effects o, 
-.001 0 .001 
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Discussion 


We use five specifications of a new and innovative competition measure to estimate the 
traditional public school response to competitive pressure from the Indiana Choice Scholarship 
Program, uncovering null to small, positive changes in math and English Language Arts 
achievement and null effects on students’ probability of graduating from high school. 

Another way of interpreting these findings is that we find no evidence that increased 
private school competition is associated with decreases in student achievement in Indiana’s 
traditional public schools in the programs’ first year of operation, when 3,911 students 
transferred to the private sector by way of this school choice program, or in later years when as 
many as 32,000 students participated. These findings may help assuage the concerns of some 
choice opponents, who expressed fears of negative effects for non-choosers at the program’s 
inception. 

We do find, however, consistent evidence of small positive effects for low-income 
children. This finding holds across multiple models using five different measures of competitive 
pressure. We also find benefits for special education students in models that use the drive-time 
measure and benefits for students who are Hispanic, low-income, have limited English 
proficiency, and are in the elementary grades in models that use the competitor count measure. 
Subgroup analyses of the graduation rate impact reveal a small benefit for students in high- 
poverty traditional public schools but this effect disappears after the first year of policy 
implementation. It is also worth noting our coefficients are likely conservative estimates of 
impact, representing lower-bound effects because we are not comparing the effect of operating in 
a competitive environment to that of operating in a non-competitive environment. We are 
examining whether test scores and graduation outcomes improved more after the establishment 
of the ICSP for students attending public schools with more and more varied private school 
options than for students attending public schools with fewer and less varied private school 
options. 

The largely null effects observed here may be a result of market dominance by a 
traditional public school sector that enrolls over 1 million students, compared to fewer than 


40,000 students participating in the state-funded private school choice program. Thus, the 
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enrollment decreases associated with individual student transfers may be too small to be 
perceived as a serious threat by traditional public school teachers and administrators. To test the 
“tipping point” theory, we search for evidence of a point at which increases in student transfers 
by way of the Indiana Choice Scholarship Program predict changes in student achievement at 
nearby traditional public schools. We find little evidence to support this theory. That is, public 
schools appear to respond positively to the voucher program if they are close to a private school 
accepting at least 10 voucher students. This initial positive test score increase holds constant, 
however, even as the number of voucher recipients in neighboring private schools increases. 
That is, we don’t observe a larger response by traditional public schools that are close to private 
schools accepting a greater number of voucher students, which casts doubt on the tipping point 
theory. This finding is consistent with findings from Milwaukee where Carnoy, Adamson, 
Chudgar, Luschei, and Witte reported, “this effect seems to have been a one-time response of all 
public schools to a change in contextual conditions rather than a continuous and differentiated 
improvement based on the degree of competition in the Milwaukee school market” (Carnoy et 
al., 2007). 

However, we believe it is important to note our dosage analysis requires more 
assumptions than our primary difference-in-differences design because it leverages variation in 
the precise number of voucher-using students attending private schools in proximity to a given 
traditional public school. The unmeasured factors leading to the differences in the voucher 
student count at neighboring private schools might also be causing changes in traditional public 
school student achievement, a potential bias we don’t have to worry about in the main models. 
An example of such a factor is a worsening academic climate at the traditional public school 
owing to frequent and disruptive leadership changes, for example, which would drive eligible 
students toward enrolling in a private school of choice and also drive down achievement at the 
traditional public school they depart. We reduce the possibility of endogeneity bias in the dosage 
findings by restricting the dosage analyses to the first year of post-policy outcomes when only 
3,911 students received vouchers, statewide, but we cannot totally rule out the possibility of bias 
affecting the dosage results. Thus, the reader is advised to interpret this particular set of findings 


with caution. 


45 


Conclusion 


Our goal with this study is to measure the competitive impact of the nation’s largest 
private school choice program, an education policy that currently provides $159 million in state 
funds annually for more than 36,000 eligible students. We test if increased private school 
competition predicts changes in student achievement and graduation rates at traditional public 
schools. We also test for a tipping point at which increases in student transfers by way of the 
ICSP might predict changes in student achievement at nearby traditional public schools. We 
accomplish these goals by using an intuitively better measure than prior research in this area and 
a statewide dataset featuring more than 837,000 unique students, making this the largest 
competitive effects study of a voucher program conducted to date and the first to examine 
graduation rates as an outcome. 

One of this study’s primary contributions to the broader competitive effects literature is 
that competition is measured five different ways, where each measure leverages drive-time to the 
nearest voucher-participating private school, a unique measure that incorporates road lengths, 
intersection turn times, speed limits, and traffic data to calculate precise travel time between 
schools in minutes. Prior studies in the competitive effects literature have never before used this 
metric to quantify the competitive threat from private school choice. 

Across models that examine student math and English Language Arts test scores and 
graduation rates, we report null to small positive impacts. We find no evidence that students in 
traditional public schools have been negatively affected by the enactment and growth of the 
Indiana Choice Scholarship Program, as measured by their test score outcomes or probability of 
graduating from high school. We do find, however, consistent evidence of small positive effects 
for low-income children. These findings may offer some degree of reassurance to those who 
expressed fears of negative effects for non-choosers at the program’s inception. 

Our findings are largely consistent with prior literature in this area, which finds that 
competitive effects are approximately zero or positive, but very modest in magnitude. In states 
where stronger positive competitive effects have been observed, it may be the case that those 


programs have design features that are more likely to yield competitive effects than the ICSP or 
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it may be an errors-in-variables issue that biases their estimates of competition on student 
outcomes toward zero because the researchers are mis-measuring competition. Future studies 
could rule out the latter explanation by applying the drive-time measure used here in other states 


and contexts. 
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Appendix: Descriptive Characteristics of 


Analysis Sample, Graduation Outcome 


Outcome Measure 
Student Graduates On Time 


Student Characteristics 
Female 


White 

Black 

Hispanic 

Other Race 

Low-Income 

Special Education 

Limited English Proficiency 


Traditional Public School Characteristics 
Enrollment 


Percent White 

Percent Black 

Percent Hispanic 

Percent Other Race 

Percent Low-Income 

Principal Male 

Principal White 

Principal Black 

Principal Hispanic 

Principal Other Race 

Teacher Percent Male 

Teacher Percent White 

Teacher Percent Black 

Teacher Percent Hispanic 

Teacher Percent Other Race 

Competition Measures 

Drive-time (mins) 
Drive-time (10+ Voucher Students) 
Competitor School Count (90 mins) 
Competitor School Count (30 mins) 
Competitor School Count (20 mins) 
Competitor School Count (10 mins) 


Mean 


0.85 


0.49 
0.77 
0.10 
0.07 
0.05 
0.39 
0.13 
0.03 


1520.87 
0.76 
0.10 
0.08 
0.06 
0.41 
0.81 
0.91 
0.08 
0.01 
0.00 
0.44 
0.93 
0.05 
0.01 
0.01 
0.05 


8.25 
16.48 
101.74 
22.14 
10.30 


SD 


0.35 


0.50 
0.42 
0.30 
0.26 
0.23 
0.49 
0.34 
0.17 


942.38 
0.23 
0.16 
0.10 
0.04 
0.18 
0.39 
0.29 
0.28 
0.09 
0.05 
0.06 
0.13 
0.12 
0.02 
0.01 
0.03 


7.89 
13.71 
39.44 
20.75 
10.79 


Min 


0.00 


0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 
0.00 


53.00 
0.00 
0.00 
0.00 
0.00 
0.04 
0.00 
0.00 
0.00 
0.00 
0.00 
0.03 
0.05 
0.00 
0.00 
0.00 
0.00 


0.07 
0.38 
21.00 
0.00 
0.00 


Max 


1.00 


1.00 
1.00 
1.00 
1.00 
1.00 
1.00 
1.00 
1.00 


4930.00 
1.00 
0.96 
0.72 
0.25 
1.00 
1.00 
1.00 
1.00 
1.00 
1.00 
0.91 
1.00 
0.95 
0.21 
0.17 
0.36 


40.96 
82.18 
176.00 
77.00 
52.00 


Notes: n = 434,596 unique students, 369 unique schools; ‘Competitor school count’ refers to the number of 
participating private schools within 90 mins' drive of a given public school. 
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